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An  article  by  D.  T.  Gillespie  entitled  "A  llieoreni  for  Physicists  in  the  Theory  of  Random 
Variables”  was  published  in  the  American  Journal  oj  Physics,  Vol.  51  (June  1983),  pp.  520-33. 
That  pedagogically  oriented  article  called  attention  to  a  somewhat  unorthodox  prescription  for 
determining  the  effects  of  subjecting  random  variables  to  mathematical  transformations.  The 
article  derived  the  “random  variable  transformation  (RVT)  theorem,”  and  then  illustrated  the 
usefulness  of  that  theorem  by  developing  from  it  a  number  of  important  results  in  statistics  and 
statistical  physics.  Because  of  rather  stringent  page  length  restrictions  in  the  American  Journal 
of  Physics  (the  current  average  length  for  an  AJP  article  is  only  about  four  pages)  several 
appendixes,  commentary  paragraphs,  and  illustrative  applications  had  to  be  deleted  from  the 
original  journal  manuscript.  This  technical  publication  contains  a  compilation  of  the  major 
items  that  were  deleted,  and  therefore  serves  as  a  supplement  to  the  journal  article.  This  work 
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ADDENDUM  A.  Obtaining  Bq.  (2). 

Fourier’s  Integral  Theorem  states  that,  for  any  suitably  behaved  function  fix),  we  have 

»  0*  »  0» 

/(*)  =  (2/i)“*  I  I  /(x')exp[i^x'-jc))  ctt'ds. 

J  J  .OD 

If  we  put  f{x)=Six—xo),  then  the  x'-integration  is  easily  performed,  and  the  result  is  Eq.  (2). 

Students  of  quantum  mechanics  will  recognize  Eq.  (2)  as  the  orthonormality  relation  for  the 
eigenfunctions  (2n)~^^exp(txs)  of  the  operator  —  id/ds,  with  x  representing  the  continuously 
distributed  eigenvalues. 

ADDENDUM  B.  Derivation  ofEq.  (9). 


Although  the  argument  given  in  the  text  to  deduce  the  integral  formula  for  (hiX))  in  Eq.  (9) 
from  the  definition  of  {hiX))  in  Eq.  (8)  shows  that  Eq.  (9)  is  indeed  quite  plausible,  that  argument 
cannot  be  regarded  as  a  rigorous  mathematical  derivation.  In  fact,  a  derivation  that  is  at  once 
rigorous,  general,  transparent  and  brief  does  not  seem  to  exist.  As  somewhat  of  a  compromise,  we 
state  and  prove  the  following  theorem,  which  is  essentially  a  generalization  of  the  so-called  weak  law 
of  large  numbers.  . 

THEOREM  :  Let  X^, ...  ,X^  be  n  independent  random  variables  which  have  a  common  density 
function  P.  Let  h  be  any  single-argument  function  for  which  the  two  integrals 
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Since  we  define  hiX)  to  be  the  random  variable  whose  sample  values  are  Mx***),  A(x*^*), ... ,  where  x*^*, 
x*^\ ...  are  the  sample  values  of  X,  then  the  above  theorem  implies  that  the  limit  on  the  right  side  of 
Eq.  (8)  "converges  in  probability”  to  the  integral  on  the  right  side  of  Eq.  (9).  Notice  that  the  theorem, 
as  stated  here,  requires  the  existence  (i.e.,  finiteness)  of  the  integral  in  Eq.  (Bib).  As  one  might 
expect,  the  theorem  is  also  valid  even  if  the  integral  in  Eq.  (B1  b)  is  undefined;  however,  the  proof  in 
that  case  is  much  more  complicated.  [The  more  general  theorem  is  due  to  Khintchine;  see  K.  L. 
Chung,  A  Course  in  Probability  Theory,  Academic  Press  (1974),  pp.  109  and  169.) 

To  prove  this  theorem,  we  begin  by  formulating  and  proving  a  lemma  that  is  essentially 
a  generalization  of  the  well-know  Chebyshev  Inequality. 

LEMMA  I  (Generalized  Chebyshev  Inequality):  Let  Xs(Xj, ... ,  X^)  be  a  set  of  re  random 
variables  (not  necessarily  independent)  with  joint  density  function  Q.  Let  f  be  any  function  of 
re  real  variables  x  b(x,  for  which  the  two  integrals 

4f*  /(x)Q(x)dx  and  S*  =  [/(x)-Aff  Q(x)dx  (B3) 

J  i  ® 

exist.  Then,  for  any  a  >  0,  we  have 

Prob{|/(X)-Af|  a  aS}Sa-2.  (B4) 

Proof  of  Lemma  /.-  Let  Si  be  the  set  of  all  points  x  for  which  |  fix) —M\  ^  aS .  Then 


Prob{|AX)-Af|  >aS}  =  (  Q(x)dx  s  (  [  —  |  Q(x)dx  s 

Jji  iq 


aS 


i:.i 


fix)-M 


aS 


Q(x)  dx 


=  a 


(  \fix)~MfQix)dx  =  a-^  . 
I 


QED 


Now  we  apply  Lemma  I  to  the  case  in  which  X^, ... ,  X^  are  re  independent  random  variables 
with  the  common  density  function  P,  whence  Q(x,,...,x,,)  =  P(Xj)P(x2)  -  P(x,i),  and  f  is  the  function 


1  ” 

Ax  x  )  =  “X 

I  n  n  • 


(B5) 


i=l 


In  that  case,  the  integrals  M  and  defined  in  Eq.  (B3)  are  given  by 


M 


r  nn  II^  ^ 


and 


1  =  1 


(B6a) 


7  =  1 


2 
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#00  #00  I  1  '*  |2  « 


). 


(B6b) 


That  M  and  indeed  exist  is  a  consequence  of  the  assumed  existence  of  m  and  in  Eqs.  (Bl), 

together  with  the  following  lemma. 


LEMMA  II:  The  quantities  M  and  in  Eqs.  (B6)  are  related  to  m  and  in  Eqs.  (Bl)  by 

M  =  m  and  S*  =  s^ln .  (B7) 


Proof  of  Lemma  II:  Letting  all  integrations  run  from  —  oo  to  <»,  and  using  {P(x)dx=  1  together 
with  the  definitions  of  m  and  in  Eqs.  (Bl),  we  evaluate  the  integrals  in  Eqs.  (B6)  thusly: 


m . 


^  =  ■  S  f  /»(x,)  fl  P(x}  =  ^  i;  I  hix)P(x)  =  ^  (nm)  = 

1=1^  j=l  1=1 

s*  =  M  d.,  ■  ( *  1  i «.,)  -  n 

=  -^  I  dxj  •••  I  dx\  ^  ^  h(x.)h(Xi^)  -  2nm  ^  h(x)  +  n^mA  1 1  P(x) 

=  i  U,  U„  *"<«,'  fl  '’1^1  +  -1 X  I  f  *,  ■  f  ‘i',  '•>*,' fl  '■I',' 

n  ,=i  ^  n  ,  =  i  kt*i}=l  ^  j  =  l 

-  2n~*m  ^  I  dXj  •••  |  dx^  h(.x)  []  P(x  )  +  |  dXj  •••  |  dx^  [  ]  P{x) 

1  f  1  "  f  f 

=  I  dx .  h^(x  )  P(x  )  +  — 'S  dx.h(x  )P(x  )  \  dx.h(x  )P(xJ 


n  j  =  i 


n  ,  =  i 


h\x)P{x)  +  —  n(n— Umm  -  2n  ’m(nm)  +  m' 
n'^ 


—  2n  ^m  ^  I  h(x  )P(x  )  + 

.t-.l  •  ■  ■ 

ill 


(/x P(x)  - 


=  -  I  (/jtP(x)(A‘'(x)  — m  I 
n 


dx  P{x)\hhx)-~2mh{x)~^ 


n  ) 


dx  P{x)\h{x)-  m  1“  =  s~/n  . 


QEt) 


r. 


"*i»^**«  **•  •  •• 
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Substituting  for  f,  M  and  S  from  Eqs.  (BS)  and  (B7)  into  Eq.  (B4),  we  get 
ProbI  ^^h(X^)-m  >  osn"*^  j  S 

Letting  c=asn"*^.sothat  a “^  =  s^E~^n~‘,Eq.(B8)  becomes,  for  any  e>0, 

Prob  II  iy  ALVJ- 


(B8) 


1  =  1 


ml  s  e  I  S  s^E  * , 


or, since  Prob{A <c}=  1  —  Prob{A  2f}, 

f  I  " 

Prob  -YmX)- 

l  fi  .  ‘ 


m 


1=1 


<e  I  >  1  -  s^E~^n  *. 


(B9) 


Now  taking  the  limit  of  Eq.  (B9)  as  n-+<»,  we  obtain  the  desired  result  in  Eq.  (B2). 


ADDENDUM  C.  Derivation  of  Eqs.  (23). 


The  random  variables  and  S„2  are  defined  in  Eqs.  (22)  in  terms  of  the  random  variables 
Xi,...  ,X„;  these  in  turn  are  defined  in  Eq.  (2  Din  terms  of  the  given  random  variable  X.  Since  Xj, 
...  ,X„  are  mutually  independent,  each  having  the  X  density  function  P,  then  their  joint  density 
function  isP(aci)P(x2)  •"  P(JCn).  Thus,  Eq.  (18)  implies  that 


(3D  f  <70  ft  n  r  CD 

-*  '  *=i  *=i  ^ 


where  the  prime  on  the  product  denotes  omission  of  k-i.  Thus,  using  Eqs.  (6)  and  (10a), 

<X  )  =  I  xP(x)dx  =  (X)  =  p. 

J  ._<l> 


(Cl 


Similarly,  it  follows  that 


<X  '^)  = 


x-P<x)dx  =  {Xh 


(C5 


Also,  for  any  i  * j,  wc  have 


fr  f  r  n  f  Wf  r»  fir® 

J_r  i  ^  D  J  —  x> 


where  the  double  prime  on  the  product  denotes  omission  of  k  =  i  and  k-j.  Thus, 
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Using  the  above  results  and  the  linearity  of  the  integration  operation,  we  proceed  as  follows: 
From  Eqs.  (22a)  and  (Cl)  we  have 


1=1 


»  =  l 


1  =  1 


whence 


<*,>=«■ 

This  establishes  Eq.  (23a).  To  prove  Eqs.  (23b)  and  (23c).  we  must  first  calculate  (Xe*>; 


(X^  =  ((.-‘Xxi(a-'X  X  ))  =  a-’X  S  «,*  > 

j=l  1=1 >=1 


i  =  l 


Now  using  Eqs.  (C5)  and  (C4),  we  get 

<X^2>  _  =  n"  V  +  /<*  - 

which  establishes  Eq.  (23b).  Finally,  we  have  from  Eqs.  (22b),  (C2)  and  (C5), 

K)  =  <»!“*  i  -  X^)  =  n~‘  i  <X,^>  -  <x;) 

i=l  i=l 

=  n-^n(X^)  -  (fi-V  +1*^1  =  (X^)  -t?  -  n"V. 
Using  Eq.  (10b)  this  easily  reduces  to  Eq.  (23c). 

If  we  define  a  new  random  variable  by 

£2  ^  ^  — y  (x  -.?r  )\ 

n-l  «  n-l.t-,  '  " 

then  Eq.  {23c)  evidently  implies  that 


(C4) 


i  =  l  i  =  lj(*i»  =  l 

where  the  last  step  uses  Eqs.  (C2)  and  (C3).  With  Eq.  (10b),  this  reduces  to 

(O  = 


(C5) 


(C6) 


(C7) 


{X^)  =  o2. 

Some  writers  prefer  to  define  the  n-sample  mean  of  X  to  be  instead  of  Sn^-  Comparing  Eqs.  (22b) 
and  (C6),  we  see  that  as  n-»®  the  sample  values  of  both  S„2  and  £„2  approach  o2;  however, 
comparingEqs.  (23c)  and  (C7),  we  see  that  only  X„2  has  o2  as  its  exact  mean.  Of  course  if  n^l  then 
the  distinction  between  S„2  and  i’„2  ig  unimportant. 
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ADDENDUM  D.  An  Alternate  Derivation  of  the  R  VT  Theorem  for  n  =  2,  m  =  1. 


Let  the  random  variables  JT]  and  X2  have  the  joint  density  function  P(xx,X2),  and  let  the 
random  variable  ybedeflnedby  y=/(Xi^2)-  To  calculate  the  density  function  Q(y)  of  V,  let  us 
assume  that  the  equation  y = f(x\  ,X2)  can  be  solved  "nicely”  for  X2  in  terms  of  xt  and  y;  i.e. ,  let  us 
assume  that 

y  =  iff  *2  “  WxjO^),  (Dl) 

where  the  functions  f  and  h  are  one-to-one  with  continuous  first  derivatives.  Then  the 
transformation 


T. 


=  X| 


=  A*i.*2i 


(D2a) 


from  xiX2-space  to  xiy-space  is  one-to-one  and  differentiable,  and  so  is  its  inverse, 


’-1. 


X,  =x, 


X2  =  AU,,y) 


(D2b) 


Let  the  infinitesimal  area  element  dx\dy  be  the  image  under  T  of  the  infinitesimal  area 
element  dx\dx2.  Then  clearly  {the  probability  that  a  simultaneous  sampling  of  Xi  and  Y  will  yield 
a  point  inside  dx\dy }  is  identical  to  {the  probability  that  a  simultaneous  sampling  of  Xf  and  X-z 
will  yield  a  point  inside  <ixidx2}.  Letting /?(xi  o')  denote  the  joint  density  function  ofXi  and  Y,  we 
can  express  this  fact  symbolically  by 


/?(Xj,y)  dx^dy  =  dx^dx^  , 


(D3) 


where  it  is  understood  that  (xj,y)  and  (xi,x2)  are  related  by  Eqs,  (D2),  and  that  dx\dx2  and  dxidy 
are  related  by  the  corresponding  Jacobian  formula 

a(xj,x2) 


dx^dx^  = 


3(Xj,y) 


dx^dy  =  |A^(.rj.y)|<fXj(fy . 


(D4) 


Here,  hy  denotes  the  partial  derivative  of  h  with  respect  toy.  Substituting  Eqs.  (D2b)  and  (D4)  into 
the  right  side  of  Eq.  (D3),  we  find  that  the  joint  density  function  of  Xi  and  Y  is 


R(XiO')  =  P(x,,h(x,,y))l/iy(Xj,y)l . 
Therefore,  according  to  Eq.  (17),  the  density  function  of  Y  alone  is 

Q(y)  =  I  dXj  P(Xj,h(XjO'))|h^(X|,y»| . 


(D5) 


(D6) 
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Notice  that  this  result  has  been  obtained  without  reference  to  the  Dirac  delta  function. 

Now  we  observe  that  Eq.  (D€)  can  also  be  written 

Q{y)=  \  dxj  <i4P(x,.Wx,.0)|h^(x,.0ld(4-y).  (D7) 

J  —00  i  —CD 

Let  us  change  integration  variables  in  Eq.  (D7)  from  (X|,0  to  (xi,X2=Wxi,^)).  Since  X2  =  Wxi,0i 
then  it  follows  from  Eq.  (Dl),  that 

furthermore,  just  as  in  Eq.  (D4)  we  have 

Therefore,  Eq.  ( D7)  becomes  under  this  change  of  variable, 

(CD  f  ^ 

dx  dx2P(Xj,X2)5(y-Axj,X2)),  (D8) 

—  OD  *  J  — 00 

which  is  precisely  Eq.  (31)  for  the  case  n  =  2. 

Although  Eq.  (D6)  allows  Q(y)  to  be  calculated  as  a  one-dimensional  integral  without  having 
to  worry  about  any  Dirac  delta  function,  it  describes  only  one  of  several  possible  ways  of  calculating 
Q(y),  and  quite  possibly  not  the  easiest  way.  Eq.  (D8)  on  the  other  hand  offers  a  variety  of  ways  to 
proceed;  If  we  change  integration  variables  in  Eq.  (D8)  from  (xi,X2)  to  (xi,  4-/(xi,X2))  and  then 
integrate  over  we  recover  Eq.  (D6);  if  we  change  integration  variables  in  Eq.  (D8)  from  (xi,X2)  to 
(ri=f[xi^2>i  ^2)  3nd  then  integrate  over  q,  we  obtain  a  version  of  Eq.  (D6)  that  would  follow  by 
solving  y  =  f{x\,X2)  forxj  instead  of  X2;  or,  if  we  substitute  into  Eq.  (D8)  an  explicit  representation  of 
the  delta  function,  suciias  that  in  Eq.  (2),  we  obtain  yet  another  avenue  of  calculation.  Thus,  Eq. 
(D8)  is  not  only  more  symmetrical,  but  also  more  flexible,  than  Eq.  (D6). 

Notice  that  the  derivation  of  the  general  RVT  Theorem  given  in  Sec.  Ill  is  much  more 
compact  than  the  above  derivation  of  the  special  case  result  Eq.  (08) 


ADDENDUM  E.  The  Delta  Function  Change-of- Variable  Theorem. 

The  following  theorem  is  useful  when  integrating  over  a  delta  function  when  the  argument  of 
the  delta  function  is  a  complicated  function  of  the  integration  variable. 
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Theorem:  Let  A  be  a  differentiable  function  ofx  whose  only  zeros  are  at  xj,  X2, ... ,  x„,  and  let 
k'(Xi)^0  for  i=  1, ... ,  n.  Then 


MhCx))  = 


n  (Rx-x) 


(El) 


Proof:  Where /is  any  function  ofx,  let 


(oo  f  <»  '«  5(i  —  X ) 

dxfl.x)S{Mx))  and  J=  dx/ix)^  ■  ■  - . 

—  03  J  —00  .  —  i  1“  (^.)| 


(E2) 


The  quantity  J  is  easily  evaluated  as 

J 


n  r  <D 

.  _  I  i  -a 


5(x— X  ) 


dx  fix) 


^  N 


^  fix) 


(E3) 


1=1 


To  evaluate /,  let  the  x-axis  be  partitioned  by  points  oo,  oj, ....  a„,  which  satisfy  —«>  —  ao<ai<  ■■■ 
<a„  =  «>,  and  also  x,€(a,- 1,  a,).  Then 

n 

/=  Y.I . 


-i-  !■ 
1  =  1 


where 


'.’F' 

J  a . 


dxfix)S(hix)). 


(E5a) 


1-1 


Now,  since  the  argument  of  the  delta  function  in  Eq.  (E5a)  vanishes  inside  (a,  _  i,  a,)  only  at  x  — x„ 
then  we  can  also  write  I,  as 

b 


I 


=  ( 


dx  fix)  ^h(x)) , 


(E5b) 


1-1 


where  6,  _  i  and  6,  are  any  numbers  satisfying  a,  _  i  ^  6,  _  j  <  x,  <  b,  ^  a,.  Let  us  choose  the  interval 
(6,-1,  bf)  small  enough  so  that  h',  which  by  hypothesis  is  nonzero  at  x  =  x„  is  nonzero  everywhere  in 
(6,_i,b,).  In  that  case  we  can  view  the  relation  ^  =  h(x)  as  a  one-to-one  mapping  of  the  interval 
(6,_  I,  6,)  about  x  =  Xi  onto  some  interval  (c,_i,c,)  about  4=0.  It  is  then  permissible  to  change  the 
integration  variable  in  Eq.  (E5b)  from  X  to  i,  =  f[x)  by  putting  x  =  A-'(4)  and 

di,  di  di 


dx  = 


|c/4/</x(  |A'(x)|  |A'(/,-'(0)| 


Eq.  (E5b)  thus  transforms  to 


I  = 

I 


c^_,|A’(A~‘(4))| 


di,  ,  fih~\0)) 

fih-\i))m  = 


1A'(A-*(0))|  ’ 


or,  since  x,  is  the  only  zero  of  h  in  (6,  _  i,  6,), 


NWCTP6462 


Inserting  Eq.  (E5c)  into  Eq.  (E4)  and  then  comparing  with  Eq.  (E3),  we  conclude  that  I=J. 
Therefore,  we  have  shown  that 


(E5c) 


f  dxmmx))  = 

J  ^OD 


»  5(x-i.) 

I,  W 


(E6) 


for  any  function /Ix).  Now  taking  /Ix)=6(x— x'),  the  x-integrations  on  both  sides  are  trivially 
performed,  and  the  result  (apart  from  x  being  replaced  by  x')  is  Eq.  (El).  QCO 


As  an  sample  application  of  the  foregoing  theorem,  suppose  h(x)=y—x^,  where  y  >0. 
Clearly,  h(x)  vanishes  at,  and  only  at,  the  two  points  xi  =yl/2  and  X2=  —  y*^ .  Also,  since 
h'ix)  =  —  2x,  then  |h'(xi)|  =  |h'(x2)|  =  2yi/2.  Therefore.  Eq.  (El)  allows  us  to  conclude  that 


2  aix-y*^  +  tflx+y*^) 

ily-x^)  =  0(y)  — 


2y 


V2 


where  the  d-function  ensures  thaty  is  indeed  non-negative  (so  thaty*^  is  real). 


(E7) 


ADDENDUM  F.  An  Application  of  RVT Corollary  II  When  f  la  Two  to-One. 

Suppose  we  wish  to  calculate  the  density  function  Q  of  F = ({X)  =  when  X  is  uniformly 

distributed  on  [ — C| ,  C2I,  where  ci  and  C2  are  both  non-negative.  Since  in  this  case  f  is  not  strictly 
one-to-one  (e.g. ,  f  sends  both  2  and  —  2  into  4),  then  Eqs.  (39)  are  not  applicable.  But  one-to-oneness 
is  not  required  of  f  by  RVT  Corollary  11;  thus,  we  can  write  from  Eqs.  (32)  and  (11), 

Q(y)  =  I  di  (C|-f-C2)‘’*O(x-l-Cj)0(c2— x)5(y— x^).  (FI) 

i  ^<X> 

Now,  there  are  two  ways  of  evaluating  this  integral.  One  way  is  to  first  render  it  trivial  by 
making  the  change  of  integration  variable  x-*z  ~  x't.  The  inverse  of  this  transformation  can  be 
either  x=— 2l/2orx=  +  21/2,  but  this  is  not  an  ambiguity  so  far  as  the  above  integral  is  concerned; 
because,  when  that  integral  is  written  as  a  sum  of  two  integrals,  one  over  the  negative  x-axis  and 
one  over  the  positive  x-axis, 

Q(y)  =  [  dx  (Cj-HcJ~*0(x+c.)<S(y-x^)  [  dx  (Cj  +  Cj)”*  0(c2-x)6(y-x\ 

we  have  no  choice  but  to  take  x=— 2I/2  in  the  first  integral  and  x= +2i/2  in  the  second.  Thus, 
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Q(y) 


r  0 

=  (-iz-‘^<fe)(Cj+C2)-'8(-2*'®+Cj)%-z) 

# 


+  j  (+*2-’^<fe)(c,+C2)-‘e(c2-2‘^)5(y-z) 

=  [2(c^+C2)r‘  dzz-^Siy-zmi.c^-z'^)  +  d{f:^-z^\ 

=  PCCj+Cj)!'*  I  dze{z)z-^8^y-z)mcl-z)  +  OCCj^-z)]. 

i  ~OD 

The  2-integration  is  now  trivially  performed,  and  yields 


Q(y)  =  fl(y)(0(Cj‘‘-y)  +  fllCj^-ylliaiCj-t-Cjll-'y-’^. 


(F2) 


Another  way  to  evaluate  the  integral  in  Eq.  (FI)  is  to  subject  the  delta  function  there  to  the 
delta  function  change-of-variable  theorem  [see  Addendum  B].  Thus,  using  Eq.  (E7).  we  get 


r"  1  ( 5(2-0 -t-dlx+y^l 

=  j  etc  (Cj  +  Cj)  *  0(x  +  Cj)  d(c^-x)  0(y)  I - - I 

=  0(y)[2(Cj-l-C2)r‘y-‘'^  [  |  dxOlx+CjieiCj-xldlx-O) 


I  <ix0(x-HCj)0(c„-x)5(x+y’^) 

J  —00 

Performing  the  x-integration  gives 

Q(y)  =  0(y)I2(Cj+C2)l-‘y"‘^{OCy‘^+c,)0(c2-y‘'^)  +  0(-y*'^-^c,)0(c2+O} , 

2  2 

Sincey>0,  the  quantity  in  braces  reduces  to  {0(C2  — y)  +  0(c|  —y)},  and  we  again  obtain  Eq.  (F2). 
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ADDENDUM  G.  The  Gamma  Distribution. 

If  the  random  variables  X\, ... ,  (na  1)  are  mutually  independent  and  each  Xj  is  E(a),  then 

the  random  variable  Y  defined  by 

n 

Y='£x.  (G1) 

i=l 

is  said  to  have  a  gamma  distribution  with  parameters  a  and  n;  we  say  that  "Y  is  r\a,n).” 

To  calculate  the  density  function  of  Y,  we  first  observe  from  Eqs.  ( 1 2)  and  ( 1 6)  that  the  joint 
density  function  of  Xt, ... ,  X^  is 

a 

P(x  ,  ...,x)=  n  0(x  )aexp(-ax  ). 

I  n  ■  *  t  1 

i  =  l 

Thus,  by  RVT  Corollary  I,  the  density  function  Q(y)  of  Y  is 


/®  /®  n  /  ^  \ 

Q(y)  =  ‘^l"'  n  ~  s  *,) 

j=l  ' 

f*  f®  /fi  — /  r  n—i 

2  V  ‘^*„0(Vexp(-axJd(x^-  y-  ^  x 
JO  Jo  i=l  '  ' -®  *  y=l 


[y  fj"*!  - *n-2 

Q(y)  =  0(y)a"exp(-ay)  I  cfxj  J  dx^  j  ^3"  ] 

[The  function  0(y)  was  inserted  when  the  delta  function  was  integrated  out  because  the  argument  of 
that  delta  function  never  vanishes  if  y<0.|  Next  we  change  integration  variables  from  (xi,...,  x„_i) 
to  where 

ui  =y-x, 

^2  “  ~*2  ’ 


“n-l  ~  •)'~*1“*2”  *n-I  • 

Since  this  is  a  linear  transformation  whose  matrix  is  triangular  with  —  I’s  along  its  main  diagonal, 
then  the  absolute  value  of  its  Jacobian  is  unity;  furthermore,  since 
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(with  UQ^y),  then  as  the  old  integration  variable  xj  in  Eq.  (G2)  ranges  over  the  integral  [0,  uj- 1 1,  the 
corresponding  new  integration  variable  uj  ranges  over  that  same  interval.  Eq.  (G2)  therefore 
transforms  to 

Q(y)  =  0(y)o"exp(-ay)  I  |  du^_2  (G3) 

This  multiple  integral  is  easily  evaluated:  The  u„-i-integration  gives  u*„_2/l!  as  the  integrand 
for  the  u„_2*integration,  which  in  turn  gives  u^„_3/2!  as  the  integrand  for  the  u„_ 3-integration, .... 
which  in  turn  gives  ui"  “  */(n — 2)!  as  the  integrand  for  the  ui-integration,  which  finally  gives 
y"”  */(n—  1)!.  We  thus  conclude  that 


Q(y)  =  0(y)aexp(-ay) 


(n-D! 


[Y  is  Ha.n)). 


Using  the  integral  identity  Eq.  (53),  it  is  easy  to  prove  from  Eqs.  (10)  and  (G4)  that  the  mean 


and  variance  of  the  distribution  r\a,n)  are 


<y>  =  n/a  and  <y2)_<y>2  =  n/o* 


[V  is  r(a,n)l. 


It  is  obvious  from  the  definition  of  the  gamma  distribution  in  Eq.  (Gl)  that 

no,l)  =  E(o), 


(G6a) 


which  is  also  clear  upon  comparing  Eq.  (G4)  with  Eq.  (12).  However,  it  is  a  little  surprising  to 
discover,  by  comparing  Eq.  (G4)  with  Eq.  (54),  that 


n4,n)  =  xH2n). 


(G6b) 


The  asymptotic  form  of  fXa.n)  for  n-»<»  can  be  deduced  most  easily  by  using  the  Central  Limit 
Theorem:  If  Y  is  r(a,n),  then  from  Eqs.  (Gl)  and  (22a)  it  follows  that  the  random  variable  Z  defined 
by 


1  1  V  „ 


can  be  regarded  as  the  n-sample  mean  of  a  random  variable  X,  which  in  turn  is  E(a).  Since  X  in  that 
case  has  mean  a-  *  and  variance  a -2,  then  the  Central  Limit  Theorem  tells  us  that  the  distribution 
ofZ  tends  to  N(a-l,a-2/n)  as  n  tends  to  °o.  Thus,  by  Eq.  (35),  the  distribution  of  Y=nZ  tends  to 
N(na-*,  n2a-2/n),  or 

rXa,n)  -*  N(n/a,  n/a^)  as  «-»<».  (G7) 

The  gamma  distribution  plays  an  obvious  role  in  the  mathematics  of  radioactive  decay:  Thus, 
for  a  radioactive  sample  whose  decay  constant  is  r  decays  per  unit  time,  the  time  required  for  exactly 


I? 
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n  decays  is  a  random  variable  whose  distribution  is  fXr,  n).  Eq.  (G7)  would  then  imply  that  the  time 
required  for,  say,  100  decays  would  be  approximately  normally  distributed  with  mean  100/r  and 
standard  deviation  (100/r2)i/2  =  iQ/r.  Similarly,  in  a  dilute  gas  with  mean-free-path  i,  the  total 
distance  travelled  by  a  molecule  in  making  exactly  n  collisions  is  a  random  variable  whose 
distribution  is  HA-  >,  n). 


ADDENDUM  H.  Sum  of  Independent  Normal  Random  Variables. 


Let  X\, ...  ,X„ben  mutually  independent  random  variables,  and  let  Xj  be  N(/ii, 
(i  =  1 . n).  Then  by  RVT  Corollary  I,  the  density  function  Q(y)  of  the  random  variable 

y=±x^ 


(HI) 


1=1 


IS 


Q(y) 


Using  Eq.  (2)  the  delta  function  here  can  be  written 

<n  \  /a>  n 

y  -  ^  xj  =  (2ii)“*  I  dsexp(isy)  [~|  exp(-tsx.). 

j=\  '  j=l 

With  this,  Eq.  (H2)  becomes 

fOD  r  00 

dsexp(uy)  P|  j  dx.  (2noj*)“’^exp(-isx  )exp(-[x  .|^/2o  \ 

— «  y— J  ®  ^  ^  ^  J  J  J 

Changing  integration  variables  from  Xj  to  2j  =  (xj—ttj)/oj  gives 

Q(y)  =  (2n)“M  dsexpiisy)  P|  (2rt)“*^exp(-is|*  )  I  dx  exp(-iso  x  .)exp(-x*/2). 

7  =  1 

But 

r  ®  r  ® 

j  dz  exp( -iso  z  ) exp( - x ^/2)  =  2  oos(so  x  .) exp( - x ^/2) dx  =  (2re)*^ exp( — s^o ^12), 
J_^7  77  7  Jq77  77  7 

where  the  last  step  follows  from  the  integral  identity  Eq.  (48).  Thus, 

n 

dsexp(isy)  exp(  — is/i  )exp(  — s^o^/2). 


7  =  t 


Now  defining 
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3 


«a 


^o“  ^  °0  “  S  °j’ 

7=t  >=l 

the  above  simplifies  to 

QCy)  =  (2«)~*  f  dsexp(i^)exp(-t^^exp(-s^OQ/2) 

i  • 


=  (2n)  2j  <»s(sty-;<gDexp(-s  OQ/2)cfe. 
Appealing  once  more  to  the  integral  identity  Eq.  (49),  we  conclude  that 


Q(y)  =  (2aOj,2)-»/2  exp(-Iy-poP/20o'') .  (H4) 

We  have  thus  proved  that  the  sum  of  n  independent  normally  distributed  random  variables  is  itself 
normally  distributed,  with  mean  and  variance  equal,  respectively,  to  the  sums  of  the  means  and 
variances  of  the  given  random  variables. 

The  foregoing  result  tells  us  that  if  Xi  is  N(;<l,0|'^)  and  X2  is  N(/i2, 02^),  withXi  andX2 
mutually  independent,  then  the  sum  A'i  +  A’2  is  N(pi +;<2»®l*+02^)-  The  difference  X1-X2  is 
handledby  regarding  it  as  the  sum  of  Xi  and  -X2:  According  to  Eq.  (36),  -Xais  N((-l)p2. 

(— 1)2o22)  =  N(— /<2.  02^);  thus,  X1—X2  is  Nin\— 112,01^  +  02^). 

If  Xj  is  regarded  as  the  i^*’  sampling  of  X  [see  Eq.  (21)),  where  X  is  N(//,o2),  then  we  have  Hi-fi 
and  o,  =  o  for  all  i  =  1  ,...,n,  and  the  foregoing  result  implies  that 

n 

'y  X^  is  N(/i/<,  noh. 
i  =  l 

It  then  follows  from  Eqs.  (22a)  and  (35)  that,  ifXis  N(/<,  o^),  then  is  S(n-^nn,  n-^na^)  =  S(fi, 
<?lviS.  This  constitutes  an  independent  derivation  of  the  important  result  in  Eq.  (77a). 


i2/o_  2\ 


ADDENDUM  I.  Orthogonal  Transformations 

Following  the  classical  e.xposition  of  Goldstein  [see  Ref.  7|,  we  review  here  some  properties  of 
orthogonal  transformations  that  are  invoked  in  Applications  8  and  10. 

The  linear  transformation  from  2ii2  "^n-space  to  u\U2  -Un-^P&ee, 

n 

M  =  'S  a  z  (i=l,...,n),  (11) 

I  ^  ,j  j 

j  =  l 


is  said  to  be  orthogonal  if  and  only  if 
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'r. 


1^  =  1“/- 

i=I  »=l 

Substituting  Eq.  (11)  into  the  right  side  of  Eq.  (12)  gives 

n  n  n  n 

is 

1  =  1  i  =  ij  =  u  =  i 

from  which  it  is  seen  that  Eq.  (12)  holds  if  and  only  if 

n 

=  ^ . 

( =  1 


a2) 


(13) 


where  Sjk  is  the  Kronecker  delta  symbol.  This  is  one  form  of  the  orthogonality  condition,  although  it 
is  not  the  form  that  we  require  in  Applications  8  and  10. 


Let  the  transformation  inverse  to  Eq.  (ID  be 


>=  0=1 . n). 


i  =  l 


Inserting  this  into  Eq.  (II)  gives 


(14) 


'i 

s 


“i=  S  S  %«'>*“*  • 

;=!*=! 


which  can  evidently  hold  for  all  Ui  only  if 


Now  consider  the  double  sum, 


5"  a  a’  -  S  ,  (i,lt  =  l,...,n). 

^  ij  jk  ik  ’  ’  ’ 

J  =  1 


n  n 

yjana’. 


(15) 


If  we  evaluate  this  by  summing  first  over  i  with  the  help  of  Eq.  (13),  the  result  is  a\<.  But  if  we  sum 
first  over  j  with  the  help  of  Eq.  (15),  the  result  is  a/*.  We  conclude  that 


Substituting  this  into  Eq.  (15)  gives 


^'kl  =  °lk 


V  a  a,  =5  .  (i,fe  =  l,...,n), 

^  tj  kj  ik  •  .  • 


(16) 


which  is  the  form  of  the  orthogonality  condition  invoked  in  Eqs.  (69)  and  ( I07a). 


(17) 
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The  Jacobian  of  the  transformation  in  Eq.  (11)  is  clearly  equal  to  the  determinant  of  the 
matrix  A,  where  (\)y»ay : 


a(u  ....  ,u  ) 

IT - 

Denoting  the  inverse  of  A  by  A  ~  *  and  the  transpose  of  A  by  A^,  we  have 


(18) 


i  =  |aa-‘|  =  |aaT|  =  |a||aT|  =  |a|2. 


(19) 


Here,  the  second  equality  follows  from  Eq.  (16),  the  third  equality  follows  from  the  fact  that  the 
multiplication  rule  for  matrices  is  the  same  as  the  multiplication  rule  for  determinants,  and  the  last 
equality  follows  from  the  fact  that  transposition  does  not  alter  the  value  of  a  determinant. 
Combining  Eqs.  (18)  and  (19)  gives 


a(Uj,...  ,u^)  I 
^^1 . ^ 


(110) 


Eq.  (12)  says  that  orthogonal  transformations  preserve  length,  and  Eq.  (110)  says  that  orthogonal 
transformations  preserve  volume. 


ADDENDUM  J.  Asymptotic  Equivalence  of  the  Poisson  and  Normal  Distributions 

Using  an  argument  similar  to  that  given  by  Present  (see  Ref.  9),  we  show  here  that  the 
Poisson  distribution  with  a  large  mean  can  be  approximated  by  the  normal  distribution  that  has  the 
same  mean  and  variance.  This  fact  was  invoked  in  our  proof  of  the  chi-square  theorem  [cf.  Eqs.  (95) 
and  (96)]. 

The  discrete- variable  Poisson  probability  function  with  mean  and  variance  a  is 

P(3c)  =  a^e-“/x!  (x  =  0,l,..).  (Jl) 

We  consider  here  only  the  circumstance 

a  ^  1,  (J2) 

in  which  case  P{.x)  differs  appreciably  from  zero  only  if  x  satisfies 

I  jc  — a|  a.  (J3) 

If  a  satisfies  Eq.  (J2),  then  any  x  satisfying  Eq.  (J3)  will  be  very  large  compared  to  unity;  we  can 
therefore  invoke  Stirling’s  formula, 

x!  =»  (2nx)‘''‘*r'e“*  (x  S>  1),  (J4) 
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and  so  approximate  Eq.  (Jl)  by 

P(x)  (2nx)  ~  (a/x)*  e* 

»  (2na)”^^(x/a)~*  c*”®, 

where  the  last  step  follows  since  1.  Rearranging  and  taking  logarithms  gives 

iogl(2fra)‘^ P(x)l  *  — xlog(x/a)  +  (x-a), 

or,  defining  e^x—a, 

logl(2rea)'^  P(x)l  =«  — (c+o)  logtl  +  (c/a)l  +  e  . 

Eq.  (J3)  implies  that  |e/a|  <  1,  so  we  can  expand  the  logarithm  on  the  right  and  retain  only  the  lowest 
order  terms  in  e/a: 

log{(2na)‘'2p(x)J  -(c  +  a)I  (r/a)  -  i(c/o)2  +  ...)  +  £ 

=“  -iie'^/a)  *  -(x-a)^/2a. 

We  therefore  conclude  that,  when  af>  I,  the  Poisson  function  in  Eq.  (Jl)  can  be  approximated  by 

P(x)  »  (2na)  ~  exp( — (x  -  a)^/2al  (J5) 

in  the  region  where  it  differs  appreciably  from  zero.  The  quantity  on  the  right  of  Eq.  (J5)  is  of  course 
the  density  function  of  the  normal  distribution  with  mean  and  variance  a. 

I 


1 

k 


